Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation step #113

Merged
merged 2 commits into from
Aug 10, 2024
Merged

Validation step #113

merged 2 commits into from
Aug 10, 2024

Conversation

empEvil
Copy link
Contributor

@empEvil empEvil commented Jul 16, 2024

The idea here is to add the option for a stage to test if the input it is supposed to run on actually have the data-structure the stage expects.
The idea is that to any stage you can add a method validate that will be run before running the main run method of the stage. The idea is that the user can then write their own validation for each stage. If no validate step is done, it should just pass it by, and continue to the execute part.
If an error is found in the validation it should raise an error and prevent an attempt to run the full stage and thereby waste lots of time.
An example for use case would be in TXPipe where the stage TXTwoPoint requires the use of a NofZ file, this file is only called at the end, and need to be structured in a specific way. The structure is something that has changed, and hence older files might not work, But you won't know until after several hours.

@joezuntz joezuntz marked this pull request as ready for review July 19, 2024 15:59
Copy link
Collaborator

@joezuntz joezuntz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor changes here only

ceci/stage.py Outdated Show resolved Hide resolved
ceci/stage.py Outdated Show resolved Hide resolved
@joezuntz joezuntz merged commit a3f65f8 into master Aug 10, 2024
6 checks passed
@joezuntz joezuntz deleted the Validation branch August 10, 2024 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants